Learning nested systems using auxiliary coordinates

Optimising nested functions using auxiliary coordinates

ParMAC: distributed optimisation of nested functions, with application to learning binary autoencoders

Many powerful machine learning models are based on the composition of multiple processing layers, such as deep nets, which gives rise to nonconvex objective functions. A general, recent approach to optimise such “nested” functions is the method of auxiliary coordinates (MAC) (Carreira-Perpiñán and Wang, 2014). MAC introduces an auxiliary coordinate for each data point in order to decouple the n...

Distributed optimization of deeply nested systems

•Many architectures share the fundamental design principle of constructing a deeply nested mapping from inputs to outputs. •Learning these architectures is challenging because nesting (i.e., function composition) produces inherently nonconvex functions. •Backprop suffers from vanishing gradients and is hard to parallelize, is only applicable if the mappings are differentiable with respect to th...

Constraint-based control of sensor-based robot systems with uncertain geometry

This short paper presents control schemes for constraintbased task specification of general sensor-based rigid-body robot systems in the presence of geometric uncertainty. The task constraints are expressed using auxiliary coordinates. The uncertain geometric parameters are also expressed using auxiliary coordinates. Based on these expressions control laws are derived to satisfy the task constr...

An Agglomeration Multigrid Method for Unstructured Grids

A new agglomeration multigrid method is proposed in this paper for general unstructured grids. By a proper local agglomeration of finite elements, a nested sequence of finite dimensional subspaces are obtained by taking appropriate linear combinations of the basis functions from previous level of space. Our algorithm seems to be able to solve, for example, the Poisson equation discretized on an...